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PATENT 



REMARKS/ARGUMENTS 

Claims 1-17 stand rejected after entry of an amendment pursuant to a request for 
continued examination. The rejection nearly verbatim reiterates the prior rejection under 35 
USC 1 03(a) based on a combination of Juvinall (cited by the Applicant) with Chen and Shaw. 
The Applicant respectfully submits that the Examiner has misapplied the references and has 
misconstrued the limitations of the claims. To further clarify the distinctions, claims 1 and 9 
have now been amended. 

Due to the complexity of this technology and the evident difficulty understanding 
the distinctions between the invention and the prior art, the Applicant requests full and specific 
consideration of the arguments and of the expert opinion submitted herewith and made of record, 
including claim amendments. If in the opinion of the Examiner, the language of the claims and 
the arguments presented still do not fully satisfy the Examiner nor address the Examiner's 
concerns respecting limitations argued, and in order to avoid need for higher administrative and 
judicial review, the Examiner is invited to call the undersigned, collect, at the number given 
below to address such concern. 

By this response, the declaration of Dr. John F. Gustafson has been submitted to 
support the patentability of the invention as claimed. Dr. Gustafson is a world-renowned 
authority in this field. Other experts are also prepared to submit opinions. However, only one 
expert opinion is supplied at this time. 

Paraphrasing, Dr. Gustafson submits that the system and method as recited in the 
claims (the concept of two-layered processing, with object independent per-frame processing of 
images without need for access to external memory for image data using parallel processors 
associated with individual pixels, and object dependent symmetric multi-processing in the 
second layer) is a new paradigm worthy of patentability. He is also of the opinion that Juvinall 
does not stand for the propositions advanced by the Examiner. He is also of the opinion that the 
Chen and Shaw references do not supply deficiencies as asserted by the Examiner, and he does 
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not find that the combination of the references makes the claimed invention obvious to one of 
skill in the art. 

With respect to claims 1 and 9, the Examiner reasoned that: 

1) "It would have been obvious to one skill [sic] in the art at the time of the 
invention to employ Chen et al teachings to Juvinall to clarify the design of the post-processing 
engine. In doing so, the design structure of this post-processing engine provides the optimal use 
of computation (column 7, lines 15-20) as well as performing two simultaneous 2-dimensional 
cross correlations of data sets (column 8 lines 7-10)." 

2) "It would have been obvious to one skill [sic] in the art at the time of the 
invention to employ Shaw et al teaches [sic] to Juvinall regarding where parallel processor 
process [sic] data with out the use of external memory to avoid memory bandwidth restriction. 
Memory management is a time consuming process that requires extra memory management 
hardware. This would further require more cost to the system without improving the efficient 
[sic] of the system of a parallel system. The motivation to combine these teaches [sic] such that 
this will automatically scale and conform to available bandwidth (abstract) [sic]." 

By this response, the Applicant renews all previous arguments, particularly those 
filed October 1 1 , 2007, and respectfully traverses the characterizations of the relevance of the 
prior references. 

Moreover, the Applicant respectfully traverses the conclusions of the Examiner as 
to the obviousness of the cited combination of references. The Applicant also respectfully 
observes that the conclusions, particularly the conclusion herein listed as 2) with respect to Shaw 
and the motivation to combine the reference with the other references, are not well reasoned. 
Shaw is said to stand for the proposition not to use external memory. As herein after explained, 
that is inadequate motivation to combine the references, and in any case it does not supply the 
deficiencies of the other references. 

The Applicant points out that the present invention must be understood as being 
directed to a new type of architecture. That architecture is not shown or disclosed in any of the 
cited patents in any combination. This characterization is confirmed by the declaration of Dr. 
Gustafson. 
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The subject matter of Juvinall, which was specifically cited by the Applicant, is 
the type of prior art with deficiencies that the present invention addresses and overcomes, as 
discussed in the background of the invention. None of the art of record discloses computational 
structures or processing: 1) on a per-frame basis with one processor per pixel and without access 
to external memory for image storage, in combination with 2) N-way symmetric multi- 
processing image processing at the object level. 

The Examiner has relied on Juvinall, column 2, lines 50-55 as an enabling 
teaching of the invention. In fact, the cited excerpt, when read in context, teaches away from the 
present invention. Reference is made to Juvinall, column 2, line 60 to column 3, line 2, which 
states: 

"By controlling operation of these electronics, the mater computer obtains pixel 
data from the camera and stores such data by pixel in memory, retrieves pixel data from memory 
and loads such data by pixel into the systolic array processor such that each of the plurality of 
one-bit processors retrieves and operates on one byte of pixel data^ returns data from the systolic 
array processor to the memory, and retrieves pixel data from memory and loads such data into 
the data dependent processor for non-sequential and/or data dependent image processing. " 
[emphasis added] 

This methodology is contrary to the teachings of the presently claimed invention, 
wherein image data is retrieved from and stored on a per-frame basis in registers integral with 
the image processing engine. The present invention does not retrieve from and write to external 
memory in the processing operations on object-independent data. Moreover, the nature of the 
claimed architecture means that the very need for external memory is minimized. 

In the Specification, the Applicant previously addressed the deficiencies of the 
prior art, to which Juvinall and Chen belong and represent. Two key features are not disclosed 
or taught: First, processing of data at the pixel level (one processor per pixel) on a per-frame 
basis without need for accessing external memory, and, second, processing of data at the object 
level with an SMP. To the second point, the Applicant respectfully disputes the Examiner's 
inferences from the prior art. To the first, the claims make it clear there is no memory access in 
the processing. Data found in integral registers associated with the central processors are 
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processed without any access to the external memory. This is a fundamental and economically 
significant difference between this and the teachings of the prior art. 

The deficiencies in the teachings of each of the references are herewith addressed. 

1 . The Examiner contended that Juvinall discloses that the pixel processor 
array receives image data as a full frame and processes it as such. That is not true. The Juvinall 
system reads the frame data into frame memory, and then the pixel processor array reads them 
block by block (or block-parallel, depending on how many pixel processors are available), and 
then the pixel processor processes the blocks such that pixels are processed sequentially within a 
block. There are a number of fundamental differences between Juvinall and the claimed 
invention. The claimed invention uses one processor per pixel, even if the frame data is 
multiplexed. Juvinall uses blocks. Juvinall describes a 1 -bit processor. The present invention 
has a processor per pixel that uses as many bits as there are in the color representation, typically 
8 bits per color. The present invention also uses the registers within the processor to hold image 
data for T-2, T-l and T. Juvinall uses a block RAM external to the processor to hold pixel data 
of a block, which in turn is a part of a frame. The claimed invention can and does traverse block 
boundaries for multiplexed frames. Traversal of block boundaries is not mentioned in Juvinall, 
nor is it required by the process described. JuvinalFs MPP SIMD systolic array is based on a 
multitude of traditional CPUs, with I/O and external memory, each requiring and relying on 
common DRAM access for code and image data as well as for swap and temporary data. 
Specifically, Juvinall describes the processing of image data such that data is streamed into 
DRAM of the system, and then this frame data is fetched from DRAM block by block into the 
block RAM within the plurality of pixel processors, and is transferred back from there to DRAM 
(to complete the frame data), and only then from there into the next layer. In effect, the common 
memory is the performance limiting factor in JuvinalFs patent, and the present claimed invention 
was explicitly made to circumvent exactly these problems and limitations. 

In the present invention, the architecture streams the data from the image sensor 
or the camera into the first layer processor array, and from there directly into the second layer 
processor. There is no need for image data to be stored anywhere outside the processing 
architecture for the first layer; all data is streamed in and out. A frame buffer between the 
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camera and the first layer processor only needs to be deployed in this architecture if the entire 
frame is comprised of multiple frames from multiple cameras, or if realignment of frame data is 
required because the data is streamed in via unsynchronized high speed serial links from a high- 
speed camera (a camera with 1 0,000 frames per second or more). 

2. The Examiner states that Juvinall does not teach that the second layer 
processing is carried out in an SMP. That is correct. In fact, Juvinall suggests the second layer 
processor array to be another MPP SIMD. 

3. The Examiner then states that Chen discloses an N-way SMP system to 
perform the second layer processing, and that Shaw teaches that such processor uses no external 
memory. Both statements are incorrect. Chen does not disclose an N-way symmetric multi 
processor system. The assertion is based on Parimics' claim - it is not Chen's disclosure. Chen 
does not disclose a second layer to be an SMP. In fact in column 1 1 , beginning at line 45, Chen 
very explicitly refers to an MPP SIMD: 

"Each Two Stage Processor... has its own Cache RAM... which supplies its data; 
hence the "Multiple Data "pan of SIMD. Cache RAMs ... can be multiport RAM, a set of bank- 
switched single-port RAMs, or a mixture of single-port and multiport RAMs. " 

4. The Shaw reference has been cited for the proposition of the benefit of 
avoiding external memory. The Examiner cited Shaw column 28, lines 40 to 55 as disclosing the 
claimed feature. Closer examination reveals that this is actually a claim that is directed to an 
external memory element that supports separate parallel processing neighbors. This has nothing 
to do with the teachings or claimed architecture of the present invention. The Examiner asserts 
that Shaw teaches the use of internal memory for image storage. That is incorrect. Shaw merely 
teaches that instruction data is held within the processors-something that every CPU does today. 
In Figures 1 OA and 1 0B, Shaw very clearly refers to instruction memory, and not to image data. 
This is equivalent to an I-Cache (instruction cache) of any modern processor. It has nothing to 
do with image data being held inside registers of a plurality of pixel processors. Thus, there is 
nothing suggesting anything relative to the fundamentally different architecture as claimed. 

The Examiner concludes with reasoning for the combination of the three 
references that is not supported by attempts to combine the references: 
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"7/ would have been obvious to one skill [sic] in the art at the time of the 
invention to employ [sic] Chen et al teachings to Juvinall to clarify the design of the post- 
processing engine. In doing so, the design structure of this post-processing engine provides the 
optimal use of computation (column 7 lines 15-20) as well as performing two simultaneous 2- 
dimensional cross correlations of data sets (column 8 lines 7-10). 

"It would have been obvious to one skill [sic] in the art at the time of the 
invention to employ [sic] Shaw et al teaches [sic] to Juvinall regarding where parallel processor 
process [sic] data with out the use of external memory to avoid memory bandwidth restriction. 
Memory management is a time consuming process that requires extra memory management 
hardware. This would further require more cost to the system without improving the efficient 
[sic] of the system of a parallel system. The motivation to combine these teaches such that [sic] 
this will automatically scale and conform to available bandwidth. [sic] " 

While this statement is difficult to interpret, it is inferred that the Examiner means 
that Shaw, which discloses internal instruction storage, together with the observation that since 
memory requires management hardware, memory storage is to be avoided. 

It is a major change and not a clarification of the post-processing engine to 
change architecture from an MPP SIMD to an SMP. Moreover, the MPP SIMD architecture 
herein claimed is not at all conventional. Shaw relates to internal instruction storage, not internal 
image data storage in an image processing engine. The second layer of the presently claimed 
invention is an SMP. No claim is made to use of an SMP without external memory. Second 
layer processing occurs in an SMP with N DFT engines and N Matrix Multiplication engines, 
and they may in fact use external DRAM to hold the preprocessed frame data, intermediate data, 
transient data and path and trajectory data, since the nature of the data is not prone to access 
collisions and do not require the complex memory management propounded by the Examiner. 
Consequently, the Examiner's argument and conclusion fail to address any conflict with the 
claimed invention. 

However, it is not obvious to one skilled in the art to use an MPP SIMD without 
the use of external memory in the first layer of image processing, and an SMP with external 
memory for the second layer of processing. 
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The Applicant respectfully traverses the reasoning that the three cited references 



can be combined to yield the claimed invention. Even if combined, they do not teach the 
claimed invention. The claimed architecture may be considered to be of the so-called Harvard 
structure. The cited art is all of the so-called von Neumann architecture. (Juvinall explicitly 
states that it is of the von Neumann model.) Therefore, Juvinall and Chen do not lend 
themselves to any obvious combination, and certainly not a combination that points to the 
claimed invention. If one were somehow to combine the teachings of the references, it would 
only aggravate the memory i/o problem, a problem which the present invention addresses and 
overcomes. 

Referring to claims 2-8 and 10-17, the Applicant has presented claims with more 
particularity that also distinguish over the prior art. 

The Applicant, as supported by the opinion of a recognized expert in the field, 
respectfully submits that the cited prior art has been misapplied to the invention as claimed. 



In view of the foregoing, Applicants believe all claims now pending in this 



Application are in condition for allowance. The issuance of a formal Notice of Allowance at an 
early date is respectfully requested. 

If the Examiner believes a telephone conference would expedite prosecution of 
this application, please telephone the undersigned at (650) 326-2400. 



TOWNSEND and TOWNSEND and CREW LLP 

Two Embarcadero Center, Eighth Floor 

San Francisco, California 941 1 1-3834 

Tel: (650) -326-2400 

Fax: (650) -326-2422 

Attachments 

KRA:deh 

61200768 vl 



CONCLUSION 



Respectfully submitted, 




Kenneth R. Allen 
Reg. No. 27,301 
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